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A minimal model of polychronous groups in neural networks is presented. The model is com- 
putationally efficient and allows the study of polychronous groups independent of specific neuron 
models. Computational experiments were performed with the model in one- and two-dimensional 
neural architectures to determine the dependence of the number of polychronous groups on various 
connectivity options. The possibility of using polychronous groups as computational elements is 
also discussed. 



I. INTRODUCTION 

Significant progress has been made in understanding 
the human brain over the past half century. The behav- 
ior of individual neurons has been studied extensively, 
using both experimental and computational methods, to 
the point where science can explain not only the charac- 
teristics of the various neuron types within neural net- 
works, but can also give a detailed account of the mech- 
anisms within the neurons themselves that cause these 
behaviors. Despite this progress, there is still a huge gap 
in our understanding of how these low-level mechanisms 
eventually result in the high-level cognitive functions of 
the brain. 

One phenomenon whose understanding may help 
bridge this gap is polychronization, an idea that was in- 
troduced by Izhikevich in 2006 [l(|. In a network with 
interconnection delays, two neurons may fire at distinct 
times, yet have their spikes arrive at a common post- 
synaptic neuron simultaneously due to the difference in 
connection delays. This phenomenon is termed polychro- 
nization. In addition these neurons plus the stimulated 
postsynaptic neuron may have their output spikes ar- 
rive simultaneously at still other neurons, causing further 
neural activity. The set of neurons in this chain reac- 
tion is called a polychronous group, which we sometimes 
shorten to polygroup. 

Polychronization is similar to the phenomenon of syn- 
fire chains [H] [3]. However synfire chains appear when 
the neural network has synaptic connections with iden- 
tical delay times, whereas polychronization occurs when 
there is a spectrum of connection delays between neurons, 
and is more like the idea of a synfire braid mentioned by 
Bienenstock. It has been suggested that synfire chains 
form the basis of learning in the neocortex |6j , while oth- 
ers have explored the information processing aspects of 
such chains @ ■ The focus of this paper is on neural net- 
works with transmission delays between neurons, a nec- 



essary condition for the appearance of polychronization. 

Precisely timed spatiotemporal patterns have been ob- 
served experimentally both in vivo and in vitro 0- 
Although these experiments seem to provide evidence for 
the existence of polychronous groups in the brain, it is an 
open question as to whether such observed activity can 
be accounted for by surrogate data generation. While 
detection of polychronous groups in theoretical models is 
straightforward, the lack of full network data in experi- 
mental situations makes their observation problematic. 

Izhikevich noted that the number of polychronous 
groups far exceeded the number of neurons in the sys- 
tems he studied. This observation led him to hypothesize 
that polychronous groups may represent memories in the 
brain, which could possibly explain the rich diversity of 
brain behavior that seemingly transcends the capabilities 
of the neurons present. 

In this paper we describe a simple neural network 
model that has a minimal number of features to sup- 
port the study of polychronous groups. We also develop 
an associated algorithm for the calculation of polygroups 
formed in the model, and apply that algorithm to various 
random networks to determine the number of potential 
polygroups in these systems. 

Additionally we describe a new form of neural com- 
putation using polychronous groups as the basic compu- 
tational elements. The simultaneous firing of two poly- 
groups can in some cases stimulate the formation of still 
other polychronous groups, leading to a cascade of activ- 
ity extending far beyond the space and time of the initial 
neural firings. This combination of polygroups into new 
polygroups suggests a higher level structure to the dy- 
namics of neural systems. 



II. DESCRIPTION OF THE MODEL 
A. Network model 
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In the original paper on polychronous groups, Izhike- 
vich analyzed a network of neurons modeled individu- 
ally by his own spiking neuron equations [jj; in addition, 
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Spike Timing Dependent Plasticity (STDP) was used to 
adjust the weights in the network. Other researchers have 
also stressed the importance of STDP in forming such 
groups [ll|]. While these features create a system that 
has certain characteristics of actual neurons in the brain, 
they are not necessary to study the phenomenon of poly- 
chronization. One of the key premises of this paper has 
been to abstract the system to the bare minimum features 
necessary for studying the pure computational concepts 
of polychronization and polychronous groups. 

A simple digraph with connection delays is sufficient 
to model the essential features of polychronization. A 
neuron model that fires a spike when the sum of its inputs 
reaches a fixed threshold is used for the nodes of the 
digraph. Connections between neurons are lossless, and 
each has a fixed, integer delay associated with it. Discrete 
time is used in the model with the same integer scale. All 
connections are excitatory in the basic model. 

The model assumes that if a neuron receives two or 
more simultaneous input spikes it will activate and fire 
its own spike. A system in which a single input spike 
causes a neuron to fire cannot be particularly interest- 
ing, since all that has happened in computational terms 
is that the spike has been delayed. Requiring a large 
number of simultaneous spikes for activation is more re- 
alistic in terms of modeling the human brain; it has been 
estimated that it takes 20 to 50 presynaptic spikes arriv- 
ing within a short time window to cause a postsynaptic 
spike in the human brain However such a system 
would be far more difficult to analyze, and is simply not 
necessary for understanding the fundamentals of poly- 
chronization. Hence, requiring two spike arrivals is the 
simplest and most tractable arrangement that will yield 
computationally rich behavior. 

To build a network in which to search for polychronous 
groups, we first choose N, the number of neurons in the 
network, and arrange these N neurons in a circular array 
(i.e. a linear array with periodic boundary conditions). 
To choose the interconnections between these neurons 
two parameters are used, 1) a fixed number of input 
connections per neuron to, and 2) a radius r of nearest 
neighbors of each neuron from which connections may be 
selected. When selecting input connections the neuron 
itself is excluded since we do not want self-connection. 
In our initial models, once the connections are set, each 
is assigned an integer delay chosen randomly from the 
range [d min ,d max ], where d mm and d max are parameters 
of the model. 

Notice that once the neural topology is fixed, the set 
of polychronous groups within the network is also fixed. 
The network itself can be studied to determine what poly- 
groups are inherent within it, irrespective of any specific 
dynamic considerations. 

An example of a polychronous group is depicted in fig- 
ure [TJ The vertical axis labels neurons and the horizontal 
axis shows time. The circles mark points at which spe- 
cific neurons fire, and the lines show the travel of spikes 
from left to right from one neuron to another. In this 
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Figure 1: A depiction of an example of a polychronous group. 

example the two initiating neurons are neuron 1 which 
fires at t=0, and neuron 3 which fires at t=l. Spikes 
from these two neurons arrive at neuron 2 at time t=2, 
causing it to fire (this implies that the delay from neuron 
1 to neuron 2 is 2 time units, and the delay from neuron 
3 to neuron 2 is 1 time unit). Spikes from neuron 3 and 
neuron 2 arrive simultaneously at neuron 4, causing it to 
fire at t=3. Finally, spikes from neuron 1 and neuron 4 
arrive at neuron 2 at t=4, causing it to fire again. 

B. Finding Polychronous Groups 

A polychronous group is determined by the indices of 
its two initiating neurons and the times at which they 
fire. The first step in our search for polygroups is to scan 
through each possible pair of neurons and examine each 
pair to see if it could initiate a polygroup with an appro- 
priate choice of firing times. For a system with N neurons 
we can form N 2 ordered pairs; however the neurons must 
be distinct and their order is unimportant, so the actual 
number of pairs we need to examine is (A 2 — N)/2. For 
each pair of neurons we must also choose the times at 
which they fire. We are of course only interested in situ- 
ations where these two neurons will cause another neuron 
to fire; for this to happen they must both have output 
connections to the same neuron. If such a common post- 
synaptic neuron exists, it is always possible to choose 
the initial firing times for the pair so that the postsy- 
naptic neuron receives spikes from them simultaneously. 
The times are relative, allowing us to choose the earliest 
firing time to be t — and to choose the other time ac- 
cordingly. For each pair of neurons being considered, we 
must examine all possible connection pairs for all com- 
mon postsynaptic neurons. 

The procedure above finds two neurons that stimu- 
late a third neuron to fire, but by definition to have a 
polygroup at least one other neuron must also receive 
simultaneous spikes. The next step in the algorithm is 
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to search for additional firings by allowing the system to 
evolve. The evolution of the system can be calculated ef- 
ficiently by creating a matrix of spike arrival counts (see 
expression Q] below) . The rows are numbered from to 
N — 1 corresponding to the N neurons in the system; 
the columns are numbered from to t max , the maximum 
time to which the simulation is run. The matrix entry 
q n ,t represents the number of spikes that arrive at neu- 
ron n at time t. Initially we set all matrix entries to zero, 
except for the two initial nodes which we set to 2 at the 
appropriate times (this simulates these neurons receiving 
2 input spikes, so that they will fire during the simulation 
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To run the simulation we start at the leftmost column 
and look for entries with a value of 2 or greater. These 
neurons have received enough spikes to cause them to fire, 
so we look up what neurons they are connected to along 
with the associated delays to find the times at which the 
spikes arrive at the postsynaptic neurons. Using these 
numbers we find and increment the corresponding matrix 
entries. We then move to the second column and repeat 
the procedure. 

The system can be run iteratively until no firings occur 
for a period of time equal to the maximum delay in the 
system. However in some cases polygroups can continue 
firing for a very long time; in fact, polygroups can extend 
infinitely in time. For this reason a limit is placed on 
how long the calculation will be performed. If the limit 
is reached the group is flagged as being overrun so that 
subsequent analysis can take this into account. At the 
end of the calculation, any matrix entry with a value of 
2 or greater corresponds to a firing neuron. If there are 
four or more such entries, we have found a polygroup. 
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Figure 2: A plot showing the number of polychronous groups 
as a function of N, the number of neurons in the system. 
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Figure 3: The number of polychronous groups as a function 
of m, the number of input connections to each neuron in the 
network. 



III. EXPERIMENTAL RESULTS 
A. Results For One Dimensional Systems 

The parameters of the experiment that can be varied 
are: 

1. N = number of neurons in the network. 

2. m = number of input connections per neuron. 

3. r = radius of nearest neighbors of each neuron from 
which connections may be chosen. 

4. d m i n = minimum delay time. 

5. d max = maximum delay time. 



The first set of experiments was set up to determine how 
the number of polychronous groups varies as the the num- 
ber of neurons in the system is changed, holding all other 
parameters constant. Figure [2] shows the results of these 
runs. For each value of N, 30 runs were averaged to- 
gether to give the mean number of polychronous groups 
for that N. The relationship is clearly linear, with a slope 
of about 2.2. For these runs, m — 5, r — 5, d m in = U 
and d max = 5. 

For the next experiments we decided to determine how 
the number of polychronous groups varies as the the num- 
ber of input connections to each neuron changes. Results 
are displayed in Figure [3j which shows that the num- 
ber of polychronous groups increases rapidly as m is in- 
creased. For these runs, N = 100, r — 10, d min — 1, and 

dmax 5. 
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Figure 4: The number of polychronous groups as a function 
of r, the radius of nearest neighbors from which input con- 
nections are chosen. 
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Figure 5: The number of polychronous groups as a function 
of d max , the maximum delay on a connection. 



We varied r in the next set of experiments to deter- 
mine how the number of polychronous groups varies as 
the the number of nearest neighbors from which input 
connections are chosen changes. Results are displayed in 
Figure HI which shows that the number of polychronous 
groups decreases rapidly as r is increased. For these runs, 
N = 100, m = 5, d mm = 1, and d max = 5. 

Figure [U shows what happens as d max is varied. The 
number of polychronous groups decreases rapidly as d max 
increases. This is intuitively clear when one considers 
that as dmax increases, the number of possible delays 
on the connections increases and so the probability of 
finding pairs of connections with simultaneous arrivals 
becomes less. For these runs, N — 100, m = 5, r = 5, 

and djnin = 1- 

So what do these results tell us about polychronous 
groups in the human brain? It is estimated that there 



are 10 11 neurons in the brain, with to in the range of 
1000 to 10,000 connections per neuron. Connectivity in 
the neocortex has been observed to be about 10%, so a 
good rough estimate of r is 5,000 to 50,000. Experimental 
measurements of axonal delays have shown that the delay 
can be as low as 0.1 msec and as high as 40 msec [l3| 

Since the number of polychronous groups scales 
linearly with the number of neurons, we might expect 
the number of groups to be roughly on the order of the 
number of neurons. Of more concern, however, is the 
scaling relative to the values of r, to, and d max , since 
these scalings are exponential in nature. Large values 
of r and d max would tend to lower the total number of 
polychronous groups, but a large value of to argues for 
a high number of such groups. The actual result for the 
human brain cannot even be estimated with the numbers 
we have so far. 

Though the calculation of polychronous groups 
stretches the capability of current computers, it is pos- 
sible to define relatively small networks and try to ex- 
trapolate measurements on them to networks of a more 
realistic size. As a baseline we chose a system with the 
parameters N = 5000, m — 100, r — 500, d min = 1, and 
d m ax = 40, which took about 12 hours of CPU time to 
run. For this system there was a total of slightly more 
than 6.1 x 10 6 polychronous groups. If we were to esti- 
mate the number of polychronous groups for this system 
based solely on the graph in Figure [21 we would expect 
somewhat over 10,000 groups; the much larger actual to- 
tal appears to indicate that the exponential growth of the 
number of polychronous groups due to the increase of to 
overpowers the decrease brought about by the change 
due to r and d m ax- This result a gree s with that found 
by Izhikevich in his original paper [id ]. 



B. Results For Two Dimensional Systems 

Because of the brain's layered geometry, it is worth- 
while to investigate how dimensionality influences the 
availability of polychronous groups. Here we ad- 
dress whether or not extending the network to a two- 
dimensional topology affects the number of polychronous 
groups. To answer this question both one and two- 
dimensional networks were constructed using identical 
parameters. For the two dimensional model, neurons 
were located on a rectangular grid. The to connections 
to a given neuron were selected at random within a circle 
of radius r. The parameters for the ID and 2D networks 
were selected so that the same number of neurons N r 
would be included in each sub-region of radius r. 

The variation of the number of polychronous groups as 
N changed is shown in Figure H3 Both relationships are 
clearly linear, though in the two-dimensional case the 
number of polychronous groups is somewhat less. The 
slope of the ID line is approximately 1.0, while the slope 
for the 2D line is about 0.8. Parameters for these runs 
were to = 4, N r — 8, d min = 1, and d ma x = 5. 
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Figure 6: The number of polychronous groups as a function 
of N, the number of neurons in the system, for networks with 
one- and two-dimensional connectivity. The one-dimensional 
network graph is marked by small circles, the two-dimensional 
graph by plus signs. 
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Figure 7: The number of polychronous groups as a function 
of m, the number of input connections per neuron, for net- 
works with one- and two-dimensional connectivity. The one- 
dimensional network graph is marked by small circles, the 
two-dimensional graph by plus signs. 

Figure[7]shows how the number of polychronous groups 
depends on to, the number of input connections. Other 
parameters were N = 100, N r — 24, d min — 1, and 

dmax 5. 

Figure[8]shows how the number of polychronous groups 
depends on r, the radius from which input connections 
are selected. Other parameters were N — 225, to = 4, 

dmin — lj and dmax — 5. 

Figure[9]shows how the number of polychronous groups 
depends on d max . Other parameters were N = 100, to = 
4, r = 2, and d mm = 1. 

As can be seen from Figures [6] through [9l the qualita- 
tive results for one and two dimensions are similar. The 
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Figure 8: The number of polychronous groups as a function 
of r, the radius from which input connections are selected, for 
networks with one- and two-dimensional connectivity. The 
one-dimensional network graph is marked by small circles, 
the two-dimensional graph by plus signs. 
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Figure 9: The number of polychronous groups as a function 
of d m ax, the maximum connection delay, for networks with 
one- and two-dimensional connectivity. The one-dimensional 
network graph is marked by small circles, the two-dimensional 
graph by plus signs. 



actual number of polychronous groups does vary some- 
what with each of the parameters, but not significantly 
so. The net result of these studies is that changing from 
one to two dimensions does not change the essential form 
of the parametric dependencies. 



C. Choosing Connection Delays Deterministically 

In the simulations above the connection delays were 
chosen randomly within a fixed range. It is a reasonable 
assumption, however, that in actual networks of neurons 
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Figure 10: The number of polychronous groups as a function 
of N, the number of neurons in the system, for networks with 
random vs. deterministic delays. The random network graph 
is marked by small circles, the deterministic graph by inverted 
triangles. 

the time delay associated with a synaptic connection will 
be approximately proportional to the distance between 
the connected neurons. If we use this assumption in 
our simulations, how does it affect the number of poly- 
chronous groups in the network? 

Figure [10] shows how the number of polygroups varies 
with N, for both a network with random delays and a 
network with deterministic delays. Figures Qj] and [12] 
show how the number of polygroups varies with m and 
r, respectively. The essential form of the relationships 
do not change when using deterministic delays, but the 
number of polygroups in the networks with determinis- 
tic delays is significantly higher than the corresponding 
networks with randomized delays. It is not immediately 
clear why the number of groups increases when the delays 
are proportional to the distance between the connected 
neurons. 



D. How Many Pairs Form Polygroups? 

Any given pair of neurons in our neural networks may 
or may not be capable of stimulating a polychronous 
group, depending on their synaptic connections and the 
associated delays, so it is reasonable and interesting to 
calculate what fraction of the neuron pairs can actually 
form polygroups. For a neuron with m input connections 
the number of possible ways that a pair of connections 
can be chosen is given by 

m! 

2!(m-2)f (2) 

Each pair of input connections has only one timing se- 
quence with which it will trigger the neuron, so this is 



14000 



1 2000 - 



tn 1 oooo - 



>■ 8000 - 

o 

Q- 
O 

fe 6000 - 
-O 

E 

ZJ 

~Z. 4000 - 



2000 - 








2 4 6 8 10 12 14 16 18 

Number of input connections 

Figure 11: The number of polychronous groups as a function 
of m, the number of input connections per neuron, for net- 
works with random vs. deterministic delays. The random 
network graph is marked by small circles, the deterministic 
graph by inverted triangles. 
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Figure 12: The number of polychronous groups as a function 
of r, the radius from which input connections are selected, for 
networks with random vs. deterministic delays. The random 
network graph is marked by small circles, the deterministic 
graph by inverted triangles. 

also the number of ways a particular neuron can be stim- 
ulated to fire. For a system with N neurons, the total 
number of ways for neurons to be stimulated is thus 




Dividing the number of observed polygroups by this num- 
ber gives us the fraction of pairs that actually created a 
polygroup. Using the data in Figure [10] for the networks 
with deterministic delays (m = 5, r = 5), we find that 
the fraction of pairs that stimulate polygroups is roughly 
constant over all N, and is equal in this case to approx- 
imately 0.6. This may provide a lower bound for more 
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realistic systems where connections are correlated. 

IV. COMPUTATION WITH POLYGROUPS 

A polychronous group can be thought of as a sort of 
automaton; starting with just two firing neurons, an en- 
tire chain of neurons is caused to fire over an extended 
period of time. The group is simply a response to the 
initial stimulus, and in our perfect simulation world of 
discrete time and distinct spikes, the response is unvary- 
ing. In that sense, then, a polychronous group can be 
thought of as a monolithic computational element. 

When a polygroup is activated, the firing neurons 
within the group will in most cases have connections to 
other neurons outside the group. These outside neurons 
receive only a single spike and thus will not fire. We can 
envision a "cloud" of such neurons surrounding a poly- 
group in both space and time. 

If two separate polygroups are activated whose firings 
overlap in time, certain neurons in the surrounding clouds 
may receive two simultaneous spikes, one from each poly- 
group, and thus be caused to fire. Furthermore, two or 
more neurons may be activated in this manner, and their 
combined action may in turn activate a totally separate 
polygroup. The net result is that in some cases, the ac- 
tivation of two polygroups can in turn activate a third 
polygroup. 

For a given network, we can label each polygroup with 
an index i and represent an arbitrary group with the sym- 
bol d. To fully specify a polygroup we must know the 
time at which the group was activated; since the relative 
times of the activating spikes are fixed, we can choose 
the time of the first activating spike as the time associ- 
ated with the polygroup, and thus write G\ to indicate 
polygroup i activated at time t. 

If polygroup G\ fires at time t\ and polygroup Gi fires 
at time t 2 , and if the combined action of these two groups 
causes another polygroup G3 to fire at time we can 
write 

Gl 1 + Gl 2 -> Gf , (4) 

where the symbol — > is read "activates". 

Times are all relative in the system, so any time offset 
r may be added without changing the above relationship: 

G\ l+T + G% +T -> Gl 3+T , (5) 

If indeed a polychronous group represents a memory 
in the brain, then equation [4] signifies that certain pairs 
of memories are capable of stimulating a third memory. 
Equation [5] shows that the relation of these memories are 



time invariant. It is interesting, however, that the two 
stimulating memories must be activated in a fixed time 
relationship to each other to cause the third memory to 
activate. 

V. SUMMARY AND CONCLUSIONS 

We have developed a computationally efficient model 
for the study of polychronous groups, constructed on 
the principle of including only the essential features re- 
quired for such groups. An algorithm is included in the 
model to rapidly identify polygroups in the network. The 
model was used to computationally investigate proper- 
ties of polychronous group formation in various network 
topologies. 

Through numerical experiments we found that the 
number of polygroups in the network depends linearly 
on the number of neurons, holding all other criteria con- 
stant. The number of polygroups decreases asymptoti- 
cally as the radius of connectivity or the range of time 
delays increases, but grows exponentially as the number 
of input connections increases. By testing a larger sys- 
tem we found that the exponential growth of the number 
of polygroups due to an increase of input connections 
dominated over the other factors we studied. 

We conducted similar experiments comparing one- and 
two-dimensional networks, and found slight numerical 
but no qualitative differences in the results. Experiments 
were then performed in which the transmission delays 
were chosen to be proportional to the distance between 
neurons, and when these results were compared with our 
initial model we discovered that there were no qualitative 
differences, but that the number of polygroups was much 
higher for the network with the proportionally chosen 
delays. 

We also introduced the concept of computation using 
polygroups. In some cases two activated polygroups can 
cause the stimulation of a third polygroup. This opens 
up the possibility of polygroups being used as monolithic 
interacting elements in a neural system. Further work 
is required to determine the properties of this type of 
computation. 

There are still many open questions regarding poly- 
chronous groups, and we have only begun to explore 
their properties. Further measurements could prove use- 
ful, such as determining the distribution of the number 
of neurons per group under various network topologies. 
Specific examples that have recently shown a lot of inter- 
est are small world and scale free networks [3|. Inhibition 
could also be added to the neural connections, to bring 
the model more in line with the workings of biological 
neural systems. 
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